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Executive Summary 


Since spring 2020, the COVID-19 pandemic has been abruptly interrupting regular instruction 
in almost all schools in the U.S. One year later, policymakers, district administrators, and 
educators are still balancing the benefits and risks of returning K-12 students to fully in-person 
school. Many are concerned about the pandemic's disruption to students’ academic progress. 

In California, educators have been focused equally on students’ mental and emotional health, 
social relationships, and learning environment, given that many students have been learning 
remotely since the onset of the pandemic. 


Anticipating the need for schools and districts to understand their students’ social-emotional 
well-being, the CORE Districts in California developed survey instruments to collect such 
information in the 2020-21 school year. The surveys were administered remotely, as all 
participating districts offered distance-only instruction. In this report, we draw on data from the 
Student well-being and learning conditions diagnostic survey taken by 32,000 students in 
Grades 4-12 from three districts at the beginning of the 2020-21 school year. We then highlight 
results from 15,000 students who also responded to the Comprehensive interim survey a few 
months later. 


Results showed that students generally rated their learning environment at school (in 2019-20, 
before school closures) and learning environment online (in spring 2020, after school closures) 
relatively highly and consistently across grades. However, students had relatively lower ratings 

for their personal and interpersonal well-being. Students’ responses varied across grades; we 
discovered different response patterns between boys and girls, English learners and non-English 
learners, and students with and without disabilities. When connecting students’ well-being 

with academics, we found that secondary school students’ interpersonal well-being was most 
correlated with academic achievement. Middle schools at which students reported higher 
interpersonal well-being also tended to have higher academic achievement in fall 2020 and higher 
academic growth from fall 2019 to fall 2020 on average. 


Comparing responses between fall 2020 and winter 2020-21, students’ home/online learning 
environment consistently improved across grades. This is a testament to the efforts of 
administrators, educators, families, and parents to improve online learning experiences for 
students throughout the 2020-21 school year. However, likely due to the continuation of remote 
learning, students in Grades 5-12 reported that they did not like school as much in winter 
2020-21 compared with how they had felt at the beginning of the school year. This report brings 
data to bear on the areas of student well-being that are most relevant at each developmental 
stage, so that all education stakeholders are able to focus their efforts on meeting the most 
pressing needs of each group of students. 
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Student Well-Being and Learning Conditions During the Pandemic: 
Evidence from the CORE Districts 


Since the beginning of COVID-19-related school closures in March 2020, educators across 
the country have faced unprecedented challenges in continuously supporting their students. 
In addition to concerns about students’ academic learning, educators in California's CORE 
districts and the CORE Data Collaborative! have been focused equally on students’ well-being 
and on learning conditions during the pandemic. To support the need for schools and districts 
to understand their students’ social-emotional well-being (Hough et al., 2021), the CORE Districts 
developed a series of survey instruments to collect information from individual students and 
staff at the beginning of and throughout the 2020-21 school year (CORE Districts, 2020a). 


This report summarizes findings from the Student well-being and learning conditions 
diagnostic survey administered remotely at the beginning of the 2020-21 school year. It also 
highlights some initial findings from the Comprehensive interim survey, which was also 
administered remotely in winter 2020-21 after several months of distance learning. The aim is to 
provide actionable insights for K-12 educators, policymakers, and other stakeholders into how 
remote learning affected students’ social-emotional well-being and their learning environments, 
given that the primary mode of instruction in all participating districts was distance only. We 
hope this report allows students’ voices to be heard as these stakeholders consider how best to 
continue to serve students and meet their needs during this challenging time. 


Student Diagnostic Survey Overview 


The pandemic has altered students’ typical education experiences in innumerable ways. 
To maximize the support of educators as they work to understand students’ well-being and 
learning conditions during school closures, the CORE Districts establisned a survey development 
team consisting of members from CORE Districts, Education Analytics, and Policy Analysis 
for California Education. The survey development team recognized the disruptions to students’ 
mental and social-emotional health, social relationships, and learning environments, which 
became areas of focus for the student survey. The team’s survey development process built on 
best practices from the CORE Districts’ annual social-emotional learning and culture/climate 


The CORE Districts are a consortium of eight California school districts—Fresno, Garden Grove, Long Beach, Los Angeles, 
Oakland, Sacramento, San Francisco, and Santa Ana—that are recognized nationally for their collective efforts to improve outcomes 
for the one million students they serve. The consortium is the first in the country to initiate a large-scale panel survey measuring 
students’ social-emotional learning skills and incorporating nonacademic measures alongside traditional academic indicators to 
inform a holistic index of school quality. The CORE Data Collaborative is a network of local educational agencies that unites the 
eight CORE school districts with other districts across California to ensure CORE innovations and insights can impact students 
throughout the state. For more details, see https://coredistricts.org/about-us and https://coredistricts.org/our-improvement-data/ 
data-collaborative-community 
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surveys (Education Analytics, 2019). The team identified key topic areas, collected a pool of 
research-based survey items under each topic area, and selected candidate items for large group 
reviews. Throughout the process, CORE Districts collected feedback from district members 

for the survey development team. Knowing the importance of providing actionable data to 
classroom teachers and administrators at multiple levels, CORE Districts facilitated several rounds 
of reviews, focus groups, and interviews with district-level and school-level administrators and 
educators. The final student diagnostic survey includes a total of 26 survey items across four 
topic areas (a complete list of survey items is in Appendix A). Table 1 lists the four topic areas the 
survey was designed to measure, the description of each topic area, and sample items. 


Considering that schools were planning to adopt various instructional modes—including 
remote, hybrid, and in-person learning—at the beginning of the 2020-21 school year, the 
student diagnostic survey was developed to be flexible and to accommodate different modes of 
administration. The survey was designed to be used for universal screening for students in 
Grades 4-12, while avoiding questions intended to surface trauma or abuse. Such questions would 
require follow up by school counselors or medical professionals that the survey development 
team was unsure would be available at all school sites (CORE Districts, 2020b). 


Most survey items had five response options, such as (1) “Never,” (2) “Once in a while,” 
(3) “Sometimes,” (4) “Frequently,” and (5) “Always,” with (5) indicating a higher rating of the student's 
well-being and their learning environment. Two items asked students, “How often did you feel 
worried?” and “How often did you feel sad?” These items were reverse coded in our analysis, 
so that high scores on all items corresponded to positive or desirable responses. In addition to 
self-rating items using a 5-point Likert scale, the survey included three free response items asking 
students to share their perceptions with their teachers in the new school year (items 24-26, see 
Appendix A), one item where students were to check all options that apply (item 22), and one 
item on a 4-point scale (item 23). In this report, we focus on student responses from the 21 items 
on the survey that were on a 5-point Likert scale (items 1-21). We provide highlighted responses 
from the three free response items in the last section of the report. 
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Table 1. Topic Areas, Descriptions, and Sample Items on the Student Diagnostic Survey 


Ke) o)(omt-1 4-1.) 1DY-\Yot d/o) d(oya) 


Personal well-being Questions about students’ well-being in the previous 
few days or week, focusing on the degree to which 
individual students were experiencing generally 
positive or negative affect and a sense of resiliency. 


Sample item 


During the past week, how often did you 
feel happy? 


(Never, Once in a while, Sometimes, 
Frequently, Always) 


Interpersonal well-being | Questions about students’ interpersonal support in the Do you have a friend your age who helps 


offer to others as well as their connection with the 
social network from which they can receive support. 


previous few days or week, including support students | you when you are having a hard time? 


(Almost never, Once in a while, 
Sometimes, Often, Almost all the time) 


Learning environment Questions looking back, asking about students’ Last year at school, was there at least one 
(2019-20 year at school experience over the previous year at school and during | teacher or other adult in your school that 
and spring 2020 online) spring 2020 online, following school closure. really cared about you? 
(Almost never, Once in a while, 
Sometimes, Often, Almost all the time) 
About me Questions that allow students to share their What are three words that describe you? 


perceptions of their learning strengths, interests, and 
challenges with their new teacher(s). 


(Free response) 


Source. CORE Districts (2020b) 


Survey Sample 


Approximately 32,000 students in 126 schools from th 
CORE Data Collaborative answered the diagnostic survey at 


ree districts that are part of the 
the beginning of the 2020-21 


school year. The number of participants was relatively evenly spread across Grades 4-12, with 


the smallest sample in Grade 6 (n = 3,015) and the largest sam 


ple in Grade 9 (n = 3,669). Most 


students took the survey within the first month of school via remote administration. The three 


districts participated voluntarily in the survey; not all three part 


icipating districts administered the 


survey to all of their students. The proportion of Grade 4-12 students who answered the survey 
(compared to the total student population in each of the three districts) was 73 percent, 

Al percent, and 1 percent, respectively.* In order to assess how representative survey respondents 
were within the districts in which they were enrolled and in the state of California, we examined 
demographic data and Smarter Balanced Assessment Consortium (SBAC) ELA and math test 
scores (from the most recent SBAC administration, in spring 2019) for students in the survey 
sample compared to data and scores across the population of the three districts and in California. 


This comparison is summarized in Table 2. 


2 We do not have information on which schools or teachers administered the survey to their students, thus we are unfortunately 


unable to calculate the survey response rates. 


0 Student Well-Being and Learning Conditions During the Pandemic: Evidence from the CORE Districts 


Policy Analysis for California Education 


Table 2. Percentage of Students in Each Student Subgroup in California (Grades K-12), 
in Participating Districts (Grades 4-12), and in the Survey Sample (Grades 4-12) 


AYU] ofef cole] ) Survey (or live) galt) Participating Difference between Difference between 
sample student districts’ student survey sample and survey sample and 

population* population (orth co) galt-W exe) eel i s(eya) districts’ population 

Female 51.6 N/A ** 48.1 N/A** 3.5 

Asian American 46 9.3 72 -4.7 -2.6 

Black 3.3 25 11.9 -2.0 -8.6 

Latinx 85.3 54.9 69.3 304 16.0 

White S21 224 6.1 -19.3 -3.0 

Economically 774 60.7 74.3 16.7 Sil 

disadvantaged 

English learner 26.3 18.6 279 Th -1.6 

Student with disabilities 76 11.7 11.2 -4.1 -3.6 

SBAC 2019 ELA: 35:5 50.9 33.0 -154 2.5 


standard met or exceeded 


SBAC 2019 math: 28.2 39.7 254 -11.5 2.8 
standard met or exceeded 


* Based on students enrolled in Grades K-12 in the 2019-20 school year. ** Student gender is not reported publicly by the 
California Department of Education. 


Source. California Department of Education, 2019-2020a; California Department of Education, 2019-2020b; EdSource, 2019. 


The comparison shows that, across the three districts, Latinx students were overrepresented 
in the survey sample by 16 percentage points. There were also more female students and 
economically disadvantaged students as well as fewer students with disabilities who participated 
in the survey compared to these districts’ student populations. The 2018-19 academic 
performance of students who took the survey was slightly higher in both ELA and math than that 
of those students in the same district who did not take the survey. 


We further compared the student populations in participating districts with those of 
California as a whole in order to understand how representative our sample of districts 
was Compared to the state overall (see Table 2). On average, surveyed districts had a higher 
proportion of Latinx students, economically disadvantaged students, and English learners (ELs). 
Students in surveyed districts had proficiency levels of academic performance that were below 
the California state average. We urge readers to take into consideration the demographic 
characteristics and academic performance of this survey sample when interpreting the results 
presented in this report, given this is not a representative sample of California as a whole. 
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Student Well-Being and Learning Conditions Survey Results 


Exploratory Factor Analysis 


Exploratory factor analysis (EFA) is a data deduction method that helps to identify the 
underlying structure of and associations among a large group of variables (Kim & Mueller, 1978). 
Parallel analysis is a commonly used technique to determine the optimal number of factors 
in EFA by comparing the information provided by the observed data matrix to the information 
generated from a simulated matrix from a random sample of the same size (Horn, 1965; 
O'Conner, 2000). For each item, EFA computes a factor loading, which quantifies the relationship 
between each item and each underlying factor; ideally, each item loads onto one and only one 
factor with a value of 0.3 or higher. We first conducted EFA and parallel analysis to explore the 
optimal number of factors across all items, to compare the goodness-of-fit statistics from each 
factor solution, and to examine the factor structure of the student surveys® so that items could 
be grouped accordingly in the final factor solution. 


The EFA results largely aligned with the main topic areas that the survey was designed 
to measure.* To maintain consistency in item groupings across responses from the diagnostic 
survey (administered from late August through late September 2020) and the comprehensive 
interim survey (administered from late October through mid-December 2020), we omitted a few 
items in the analyses reported here that loaded differently across the surveys. We also omitted 
a few items that did not load strongly on any of the factors. In the end, we formed four groups of 
items as displayed in Table 3. 


5 In this report, we include results from the diagnostic survey administered at the beginning of the school year and the comprehensive 
interim survey administered in the winter. In addition, four districts administered the comprehensive interim survey in fall 2020, though 
we do not report the results of that survey in this report. When conducting the EFA, we aimed to find a consistent factor structure 
across all three sets of surveys. Therefore, data from all three survey administrations were used to determine jointly the factor structure 
of the survey items. The EFA was repeated within each of the three surveys and each grade level before determining the final factor 
structure. 
* Our main purpose was to expose and summarize response patterns in students’ well-being and learning environment topic areas 
over the previous year, rather than confirming the fit between the data and a hypothesized measurement model or validating the 
measurement scales. Therefore, we do not report results from confirmatory factor analysis. 
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Table 3. Four Factors and Survey Items from Exploratory Factor Analysis 


Factor h<-Vantomlaret lore (-voMC-ValeMa-t4 eLelax-Me) old (elary) 
Personal well-being During the past week, how often did you feel happy? 
(3 items) 


During the past week, how often did you feel worried? 
During the past week, how often did you feel sad? 


(Never, Once in a while, Sometimes, Frequently, Always) 


Interpersonal well-being When | need help, | find someone to talk to. 
(3 items) 


Are there students at your school who would help you if other students are being mean to you? 
Do you have a friend your age who helps you when you are having a hard time? 


(Almost never, Once in a while. Sometimes, Often, Almost all the time) 


School learning Last year at school, was there at least one teacher or other adult in your school that really cared 
environment about you? 
(5 items) 


Last year at school, did your teachers treat you with respect? 
Last year at school, did your teachers care what you thought? 


Last year at school, did teachers go out of their way to help students? 


Last year at school, did your teachers work hard to help you with your schoolwork when you 
needed it? 


(Almost never, Once in a while. Sometimes, Often, Almost all the time) 


Home/online learning When you were learning from home this past spring, how often were you able to access your 
environment schoolwork that was online? 
(3 items) 


When you were learning from home this past spring, how often did your teacher have online 
lessons with the class? 


When you were learning from home this past spring, how often did you join online learning 
lessons with your teacher? 


(Almost never, 1 day per week, 2-3 days per week, 4 days per week, 5 days per week) 


Note that three of the four factors each consists of only three items. Given this limited 
number of items, we calculated the mean of the raw responses of items within each of the four 
topics (ranging between O and 4).° 


Many of our results are based on the four topics in Table 3, though we also consider 
response patterns based on seven individual items that did not align strongly to one of the four 
topics above. These individual items are: 


Item 4. During the past week, how often did you feel hopeful? 


(Never, Once in a while, Sometimes, Frequently, Always) 


° In this report, we use the term topics to refer to the four groups of items that emerged from EFA, rather than referring to them as 
comprehensive measurement scales of psychological constructs, the term used in psychology and behavioral sciences to describe 


domains of behaviors. 
edpolicyinca.org @Q 


VA PACE 


Item 6. When | have a problem, | can come up with lots of ways to solve it. 
(Never, Once in a while, Sometimes, Frequently, Always) 

Item 9. Last year, did you feel safe at school? 
(Almost never, Once in a while, Sometimes, Often, Almost all the time) 


Item 15. Last year at school, did you get your work done right away instead of waiting 
until the last minute? 


(Almost never, Once in a while, Sometimes, Often, Almost all the time) 


Item 19. When you were learning from home this past spring, how often would you have 
someone available to help you with your schoolwork, if needed? 


(Almost never, 1 day per week, 2-3 days per week, 4 days per week, 5 days 
per week) 


Item 20. When you were learning from home this past spring, about how much time did 
you spend each day on schoolwork? 


(O hours, 1 hour, 2-3 hours, 4-5 hours, More than 5 hours) 
Item 21. In general, would you say that you like school? 
(Not at all, A little, Somewhat, Pretty much, Very much) 


Figure 1 displays the factor loadings from a four-factor EFA at each grade level. The four 
survey topics are displayed separately in each panel. Within each panel, results are displayed 
by grade, with Grade 4 appearing at the bottom of each panel and Grade 12 at the top of each 
panel. The horizontal axis shows the factor loading of each survey item which is represented 
by a dot. Dots are colored to indicate the survey topic they were designed to be a part of. Note 
that there is no set cut point for identifying a sufficiently high factor loading. We used generally 
accepted cut points in the range of 0.3 and 04 (e.g., Osborne et al., 2008) to determine whether 
a factor loading is reasonably high. For instance, in the top panel, the green dots are items that 
were designed to be part of the “Personal Well-Being" survey topic. Across all grades, these green 
items have a factor loading of 0.4 or higher. In addition, there are no green items with a factor 
loading of more than 04 on any of the other three topics (i.e., there are no green dots to the 
right of 0.4 in the other three panels), indicating that these green items load uniquely and most 
strongly together in this "Personal Well-Being” grouping. 
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Figure 1. Four-Factor Exploratory Factor Analysis Results 


1. Personal Well-Being 


Grade 


0.0 0.4 0.8 
Factor loading 


@ Home/online learning environment @ Personal well-being 
® Interpersonal well-being ® School learning environment 


Students’ Responses Across Topics and Grades 


We next examined the mean and standard deviation (SD) of responses on each of the four 
main topics and on the seven selected individual items at each grade level. Note that these are 
raw responses from students in different grades. The following results summarize the patterns 
shown in the data, but these results may not be directly comparable across grades given the 
different ages of these students and their potentially divergent interpretations of the survey items. 
Similarly, numeric responses across the four topic areas are not directly comparable because 
response options varied across items and measured distinct concepts. 
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To characterize students’ raw responses, we plotted the means of students’ ratings on 
each topic area at each grade (see Figure 2). Among the four main topics, students rated their 
school learning environment most highly and relatively consistently across all grades. With all 
items on a 5-point scale, the raw scores range from O to 4. Students gave the highest rating to 
the statement that they were treated with respect by their teachers (mean = 3.38). In contrast, 
interpersonal well-being was rated lowest across topics. Students on average responded that only 
“Sometimes” they found someone to talk to when they needed help (mean = 2.24). When asked 
"Do you have a friend your age who helps you when you are having a hard time?" 71 percent of 
12th graders responded “Often” or “Almost all the time” (mean = 2.95) whereas only 53 percent of 
fourth graders responded so (mean = 247). 


Indeed, students rated their well-being and learning conditions differently across grades. 
High school students reported relatively higher ratings on interpersonal well-being. Elementary 
students reported higher ratings on personal well-being and home/online learning environment. 
For example, when asked “During the past week, how often did you feel worried?” more than 
53 percent of fourth graders responded “Never” or “Once in a while” (mean = 2.67) while only 
31 percent of 12th graders responded so (mean = 2.06). In addition, when learning online in 
spring 2020, high school students reported that their teachers held online lessons less frequently, 
and that these students less frequently joined online learning lessons than did students in 
Grades 4-8. Older students also reported that they were less likely to have someone available to 
help them with their schoolwork in spring 2020, if needed. 


Several factors may have played a role in these varying responses across grades. As high 
school and younger students are at distinct developmental stages, they may have different 
social-emotional needs and realities affecting their well-being. For example, older students 
may have needed to take care of younger siblings and other family members, giving them less 
available time to attend online lessons. Our findings that response patterns differed across grades 
and, in particular, that students in higher grades reported lower average scores On some 
social-emotional areas, align to previous research using CORE's annual student social-emotional 
learning survey (see West et al., 2020, Figure 2). It is also worth noting that students of various 
ages may not have understood and interpreted these survey questions in the same ways. Readers 
should keep in mind these differences when considering response patterns across grades. 
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Figure 2. Mean Scores of Main Survey Topics, by Grade 


eo 1. Personal well-being oe 3. School learning environment 


o 2. Interpersonal well-being oe 4. Home/online learning environment 


In addition to the differences in mean scores, the variance of students’ ratings also varied. 
Figure 3 shows boxplots for each topic and each grade, where the median score is indicated 
by the thick black line inside each colored box; the interquartile range (i.e., between the 25th 
and 75th percentiles) is indicated by the bottom and top edges of each box, respectively; the 
5th percentile and 95th percentile are indicated by the line below and line above each box, 
respectively; and outlier responses are indicated by individual dots. Overall, students’ responses 


edpolicyinca.org & 


VA PACE 


about their home/online learning environment in the spring (SD = 0.99) and their interpersonal 
well-being (SD = 0.98) varied the most. Students’ ratings on their personal well-being (SD = 0.78) 


and school learning environment (SD = 


0.79) had smaller ranges. 


Figure 3. Boxplots of Main Survey Topics, by Grade 
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Results by Student Demographic Characteristics 


To understand whether students’ survey responses differed across demographic 
subgroups, we conducted t-tests to compare responses from students who were economically 
disadvantaged and those who were not; English learners and non-English learners; male and 
female students; students with and without disabilities; and students from various racial/ethnic 
backgrounds. The analyses were carried Out separately for each demographic subgroup, 
grade, and survey topic. Given the multiple t-tests required in these comparisons, a Bonferroni 
correction was used to adjust for potentially inflated type-! error rates. 


Somewhat surprisingly, resoonses were not significantly different between economically 
disadvantaged and noneconomically disadvantaged students in most grades and survey topics. 
Figure 4 shows that the survey responses between these two subgroups are very similar. Only 
seven (out of 36) pairs of comparisons were statistically significant. For example, economically 
disadvantaged students (vs. noneconomically disadvantaged students) in Grade 4 rated themselves 
significantly lower on personal well-being (2.73 vs. 2.86) and on home/online learning environment 
(3.12 vs. 3.55). In all other pairs of comparisons, economically disadvantaged students rated 
themselves similarly (i.e., differences were not statistically significant) or slightly higher than their 
peers (e.g., Grade 6 on personal well-being and Grade 8 on school learning environment). 
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Figure 4. Mean Scores of Main Survey Topics, by Economic Disadvantage Status and Grade 
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Note. ns = not significant (p-value > 0.05); * p-value < 0.05; ** p-value < 0.01. The vertical black bars represent the 95 percent 


confidence interval of the difference in mean scores. 
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Unlike these responses by economic disadvantage status, about half of the comparisons 
between English learner (EL) and non-EL students were significantly different. Figure 5 shows that 
ELs in Grades 5-12 rated their interpersonal well-being consistently lower than non-ELs (though 
without statistically different results, EL students in Grade 4 also responded lower on interpersonal 
well-being than non-ELs). In addition, ELs in Grades 4-6 rated their home/online learning 
environment significantly lower than non-ELs, and ELs in Grade 4 rated their personal well-being 
and school learning environment lower than non-ELs. However, ELs in high school rated their 
personal well-being higher than their peers; they also rated their school learning environment 
higher (results from Grades 9 and 11 were statistically significant). 


Given these results, educators, parents, and other stakeholders should pay special 
attention to younger EL students across all of these topics, and to interpersonal well-being 
for EL students across Grades 4-12. 
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Figure 5. Mean Scores of Main Survey Topics, by English Learner Status and Grade 
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Note. ns = not significant (p-value > 0.05); * p-value < 0.05; ** p-value < 0.01. The vertical black bars represent the 95 percent 
confidence interval of the difference in mean scores. 


Figure 6 shows that students with disabilities reported less connection with their peers 
than students without disabilities, especially after transitioning to middle school (i.e., Grades 6, 
8, 9, 11, and 12; note the wider confidence intervals for students with disabilities due to a smaller 
sample size of students with disabilities in Grade 10). Responses for younger students with 
disabilities were significantly lower on both school and home/online learning environments 
(i.e., school learning environment in Grade 5; home/online learning environment in Grades 4-6). 
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These results further emphasize the importance for educators, policymakers, and other 
stakeholders to pay special attention to supporting students with disabilities, particularly 
given the additional challenges the pandemic has introduced for these students as they 
navigate the new realm of distance learning. 


Figure 6. Mean Scores of Main Survey Topics, by Disability Status and Grade 
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Note. ns = not significant (p-value > 0.05); * p-value < 0.05; ** p-value < 0.01. The vertical black bars represent the 95 percent 
confidence interval of the difference in mean scores. 
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Our analysis also revealed some gender differences in survey responses, primarily related 
to well-being topics. Figure 7 shows that girls rated their interpersonal well-being significantly 
higher than boys across Grades 4-12, whereas boys rated their personal well-being significantly 
higher in Grades 6-12. In addition, girls in Grades 4-6 rated their school learning environment 
significantly higher; girls in elementary grades rated their home/online learning environment 
slightly higher as well, but only the differences in Grade 5 were statistically significant. 


Figure 7. Mean Scores of Main Survey Topics, by Gender and Grade 
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Note. ns = not significant (p-value > 0.05); ** p-value < 0.01. The vertical black bars represent the 95 percent confidence interval of 


the difference in mean scores. 
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Most differences in survey ratings among students in various racial/ethnic subgroups 
within each grade were not statistically significant. These graphs are included in Figures B1-B3 in 
Appendix B. 


Responses at the School Level 


Before examining differences across schools in their students’ average responses, we first 
compared how much of the variance can be explained between different schools versus within 
the same school. We found that most of the total variance in students’ responses was explained 
within school (the variance ranged between 89 percent and 99 percent). The between-school 
variance only explained between 1 percent and 11 percent of the total variance across four survey 
topics and grade levels. On average, the between-school variation accounts for roughly 4 percent 
of the total variation in students’ self-ratings. This small between-school variance means that 
most of the variation in students’ survey responses is not explained by the school the student 
was attending but by other factors. This finding is similar to that of previous research on CORE's 
annual social-emotional learning survey. For instance, Fricke et al. (2019) found consistent patterns 
that between-school variance components in the social-emotional competencies were much 
smaller than those in the SBAC scores (e.g., 4 percent in social awareness vs. 22 percent in math 
scores). The large variations in social-emotional and well-being ratings across students within 
schools could mean that schools have had a smaller effect on students’ well-being and their online 
learning environment over the past year, with more of the effect explained by factors outside 
school (or, perhaps, within specific classrooms). Schools may also be similar in terms of the topics 
that are included on the survey. Additionally, smaller between-school variance may stem from 
measurement errors in student self-ratings given that they were collected via short surveys, which 
generally provide less precise estimates of topics or competencies than do longer instruments. 


Despite the low overall variation between schools, it is worth noting that, in the areas 
of personal well-being, interpersonal well-being, and school learning environment, there were 
relatively larger variances between schools at higher grades (see Figure 8), which indicates larger 
differences among secondary schools in these areas compared to among elementary schools. 
In contrast, there were very small variations between schools with regard to home/online 
learning environment, particularly in Grades 8-12. This indicates little difference among most 
secondary schools in their students’ ability to access and participate in online schoolwork and 
online lessons in spring 2020. 
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Figure 8. Percent Variance Between Schools in the Total Variance of Students’ Responses 
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Among the 126 schools that administered the fall 2020 survey, most serve a large proportion 
of students who receive free and reduced-price lunch (FRL; mean = 76.9 percent) and who are 
Black, Indigenous, and people of color (BIPOC; mean = 95.4 percent). We compared average 
survey responses between those schools that serve 100 percent students who are BIPOC (school 
n = 43) and those that serve fewer than 95 percent students who are BIPOC (school n = 29); 
we also compared average responses between schools that serve a relatively small percentage 
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of students who receive FRL (the lowest quartile, or Q1, in the sample; school n = 32) and those 
that serve a relatively large percentage of such students (the highest quartile, or Q4, in the sample; 
school n = 32). Figure 9 shows that, in our sample, students attending schools serving a higher 
percentage of students who receive FRL and are BIPOC reported that they spent significantly less 
time each day on their schoolwork in the past spring (Item 20). Students in these schools also 
reported that they were less likely to have access to their online schoolwork (Item 16). On a more 
positive note, despite not being statistically significant, students attending these schools reported 


that they were less worried than students who did not attend these schools (Item 2). 


Figure 9. Difference in School Means, by Percentage of Students Who Were Black, Indigenous, 
and People of Color Versus Free and Reduced-Price Lunch 
Black, Indigenous, and People of Color (BIPOC) Free and Reduced-Price Lunch (FRL) 
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Note. ns = not significant (p-value > 0.05); ** p-value < 0.01. The vertical black bars represent the 95 percent confidence interval of 


the difference in mean scores. 


We next examined the variation among school-level means. Figure 10 displays school 
means within each grade band (i.e., elementary/Grades 4-5, middle/Grades 6-8, and high/ 
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Grades 9-12). For example, the range of the school means for the home/online learning 
environment topic among elementary schools was quite large; some outlier elementary school 
students reported low ratings on items related to their home/online learning experiences in 
spring 2020 (as denoted by the black dot below the blue box in the fourth panel). Response 
patterns also varied among elementary, middle, and high schools for specific items. For example, 
in comparison with older students, on average, elementary school students reported that they 
had better access to online schoolwork (item 16), that their teachers had more frequent online 
lessons (item 17), and that they joined online lessons more frequently (item 18). 


Figure 10. School Means by Grade Band 
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Connecting Students’ Self-Reported Well-Being 
with Academic Performance 


Some students who participated in the fall 2020 diagnostic survey also took the NWEA MAP 
assessment in fall 2020. In this section, we report the association between students’ academic 
performance and their well-being and learning conditions. 


In Our survey sample, about 18,000 students (55 percent) took MAP in fall 2020 in at least 
one subject (ELA and/or math). More than two thirds of the students between Grades 4 and 10 
in our sample took at least one MAP test, but very few 11th and 12th graders took the MAP test in 


our survey sample (see Figure 11). 


Figure 11. Number of Students Who Took MAP and Who Did Not Take MAP in the Survey Sample 
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We compared students’ demographic characteristics (Table 4) and students’ survey 
responses (Table 5) between those who took MAP and those who did not. Table 4 shows that 
students who did and did not take the MAP test are similar in terms of gender and disability 
status. However, more students who were Latinx, economically disadvantaged, and English 
learners took the MAP test in fall 2020. These two groups of students also did not vary much in 
their average survey responses (see Table 5). 


Table 4. Percentage of Students by Demographic Characteristic Between Those Who Took the 
MAP Test and Those Who Did Not 


DY -Vaatete]¢-) elalcomi0] ole] celele) Students who took the MAP test Students who did not take the MAP test 
Female 48.7 484 

Asian American 3.9 8.0 

Black 2.2 71 

Latinx 88.2 71.6 

White 2.9 6.1 

Economically disadvantaged 82.2 70.8 

English learner 26.7 23.0 

Student with disabilities 8.0 LS 


Table 5. Mean (and Standard Deviation) of Student Survey Responses Between Those Who Took 
the MAP Test and Those Who Did Not 


Survey topic Students who took the MAP test Students who did not take the MAP test 
1. Personal well-being 2.68 (0.76) 2.51 (0.80) 
2. Interpersonal well-being 2.45 (0.99) 247 (0.97) 
3. School learning environment 3.13 (0.76) 3.01 (0.83) 
4. Home/online learning environment 2.99 (1.04) 2.96 (1.00) 


In addition to students’ MAP test scores in fall 2020, we had access to the same students’ 
MAP test scores from the previous fall, in 2019. Using students’ academic achievement scores 
from these two time points, we were able to calculate students’ academic growth over the 
1-year period from fall 2019 to fall 2020 (please refer to Pier et al, 2021—and, more specifically, 
tO Aj29 in equation 1—for more details on the calculation of growth in MAP test scores). Table 6 
displays the mean and standard deviation of MAP ELA and math test scores in both years and 
MAP growth estimates from the same group of students by grade. Table 6 reports MAP test 
scores on the assessment's original score scale, whereas the growth estimates were standardized 
to have a mean of zero and the same SD as the fall 2020 MAP test scores within each subject 
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and grade, across the five California districts included in the fall-to-fall MAP growth sample in 
Pier et al. (2021). One of these five districts administered the student diagnostic survey and was 
thus included in our sample. Table 6 shows that students’ MAP test scores increased by 2-9 MAP 
score points between fall 2019 and fall 2020. However, compared to the other four California 
districts in the sample that administered MAP, students in lower grades (i.e., Grades 4-8 in ELA, 
Grades 4-7 in math) did not grow as much on average, whereas students in higher grades 

(i.e., Grades 9-10 in ELA, Grades 8-10 in math) grew more than average. 


Table 6. Mean and Standard Deviation (SD) of Students’ MAP ELA and Math Test Scores and 
Fall-2019-to-Fall-2020 Growth Estimates 


Student grade in MAP ELA 
ae coaes Fall 2019 Fall 2020 


Fall-to-fall growth 


4 182 14.3 191 15.0 -0.86 9.47 
5 193 14.7 198 15.3 -1.74 9.23 
6 199 14.1 203 15.5 “1.77 9.56 
7 205 45 208 16.0 -1.10 9.68 
8 209 14.5 212 15.6 -0.61 9.79 
9 214 47 217 15.9 0.27 9.29 
10 216 154 219 16.7 0.23 9.53 


Student grade in MAP math 
2020-21 


Fall 2019 Fall 2020 Fall-to-fall growth 


4 185 11.5 194 12.6 -2.28 781 
5 197 12.8 203 14.5 -174 7.53 
6 205 13.7 207 13.2 -1.99 732 
7 211 12.9 215 14.8 -0.96 7.66 
8 215 14.3 221 16.3 0.51 8.26 
9 222 16.1 227 175 0.98 8.05 
10 225 17.3 230 18.6 1.87 843 


Next, we examined the general pattern between students’ survey responses and their 
academic growth from fall 2019 to fall 2020. Overall, students whose responses were in the top 
quartile (among the top 25 percent of all students) for each of the four survey topics consistently 
experienced more academic growth in both math and ELA between fall 2019 and fall 2020 (see 
Figure 12). The difference in growth estimates was particularly large for interpersonal well-being 
and school learning environments. This finding indicates that students with higher ratings on their 


edpolicyinca.org (25 ] 


VAPACE 


interpersonal well-being and their learning environment at school before school closures 
in spring 2020 were less affected by the pandemic academically than were those who rated 
these topics lower. 


Figure 12. Fall 2019-to-Fall 2020 Growth in MAP ELA and Math Scores Between Students in the 
Bottom Quartile (Q1) and Top Quartile (Q4) of Each Survey Topic 
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The patterns between students’ ratings on well-being and learning environment and their 
academic growth varied across grades and demographic subgroups. We include these results in 
Appendix B (i.e., Figures B4—B7 and Tables B5-B6). 


To further examine the relationship between students’ well-being and their academic 
performance over the past year, we computed four sets of correlations between students’ 
performance on MAP and their ratings on the well-being and learning conditions survey: 


1. Student-level achievement correlations: Correlations between (a) students’ MAP ELA 
and math scores in fall 2020 and (b) students’ survey responses 
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2. Student-level growth correlations: Correlations between (a) students’ MAP ELA and 
math growth scores from fall 2019 to fall 2020 and (b) students’ survey responses 


3. School-level achievement correlations: Correlations between (a) students’ average 
MAP ELA and math scores in fall 2020 within a school and (b) students’ average survey 
responses within a school 


4. School-level growth correlations: Correlations between (a) students’ average MAP ELA 
and math growth scores from fall 2019 to fall 2020 within a school and (b) students’ 
average survey responses within a school 


The first two types of correlations used students as the unit of analysis. The correlations 
with MAP scores in fall 2020 can help us understand how (and if) students’ well-being and 
learning conditions in the 2019-20 school year (at school the first half and online the second 
half) were related to their test scores when they returned to school at the beginning of the 
2020-21 school year. With access to students’ MAP test scores in both fall 2019 and fall 2020, 
we were also able to examine how students’ well-being and learning conditions were associated 
with their academic growth after experiencing school closure in the spring. 


The last two types of correlations used schools as the unit of analysis, with the correlations 
weighted by the number of students within each schoolL® Student-level correlations are typically 
more straightforward to calculate and interpret; however, results at the aggregated school level 
allowed us to explore patterns among low-achieving (or low-growth) schools and high-achieving 
(or high-growth) schools. School-level correlations also dampen the random error that is more 
prevalent in student-level responses. Pearson's correlation coefficients were reported for each 
type of correlation. 


Student-Level Correlations 


The correlations at the student level with both MAP achievement and growth scores 
were relatively weak (see Tables B1-B4 in Appendix B for complete results). For most topics and 
individual items, the correlation between survey responses and MAP test scores were 0.2 or less, 
which was somewhat similar to Our previous findings of relatively weak associations between 
state summative test scores and social-emotional competencies measured by the CORE SEL 
annual survey (Wang et al., 2019; West et al., 2020). 


However, we found distinct patterns between elementary students and older students. 
For elementary students, the online learning environment topic had the strongest association 
with ELA (r = 0.21) and math (r = 0.22) test scores, but no such association is found (r = O) among 


© The weight was calculated as W = Vityary X Nyarz, WNEFE Mygry WaS the student sample size in variable one and Nyq,2 was the student 


sample size in variable two. 
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middle school and high school students. For middle school and high school students, interpersonal 
well-being was the topic that had a comparatively stronger association with ELA and math scores 

(r ranges between 0.11 and 0.13) relative to other survey topics, whereas such associations among 
elementary students were relatively weaker. In addition, several individual items had relatively 
larger associations with test scores than did others among elementary students, including 

“Last year at school, did you get your work done right away instead of waiting until the last minute?" 
(ELA r = 0.21, math r = 0.20) and “Last year, did you feel safe at school?” (ELA r = 0.15, math r = 0.13). 


The student-level correlations between survey responses and academic growth from fall 
2019 to fall 2020 as measured by the MAP test were also weak (r < 0.1). But, relatively speaking, 
the correlations among elementary students were slightly stronger than those among older 
students in topics related to personal well-being, school learning environment, and home/online 
learning environment. The only exception was the interpersonal well-being topic, which had 
a slightly higher association with academic growth among middle school students than it did 
among elementary students. 


We recognize that students’ self-ratings from a short survey are less reliable and include 
larger measurement error compared with longer assessments, such as those often used in the 
academic domain (e.g., Duckworth & Yeager, 2015). Given the COVID-19 pandemic and school 
closures in spring 2020, many unprecedented changes have affected these students’ lives. It 
may be difficult for some students to reflect on their past experiences accurately and critically. 
MAP test scores In fall 2020 and students’ growth in MAP from fall 2019 to fall 2020 also require 
different interpretation than do those from a normal year (Kuhfeld et al., 2020). All these factors 
may play a role in the relatively low correlations between students’ survey responses and 
academic performance at the student level. We thus aggregated students’ ratings at the school 
level in order to minimize the amount of random error in individual students’ responses. 


School-Level Correlations 


First, we examined school-level correlations between average survey responses and 
average academic achievement. Overall, students’ well-being and learning conditions were 
moderately associated with their academic achievement at the school level. This indicated that 
schools at which students reported higher well-being and learning conditions tended to have 
higher average academic achievement. Tables 7 and 8 provide school-level correlations between 
students’ MAP ELA and math scores in fall 2020 and students’ survey responses, including the 
four topic areas (Table 7) and seven individual items (Table 8), by grade band. There was a total of 
41 elementary schools and about 20 middle schools, but only 8 high schools in the data. Given 
the small number of high schools, we excluded those correlations in these tables. Moreover, we 
reported weighted correlations in which the correlation coefficients were adjusted based on the 
number of students within each school, so that students were equally weighted across schools. 
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We continued to discover results like those at the student level: different patterns in 
school-level correlations between elementary and middle schools (see Table 7). In elementary 
grades, all four well-being and learning condition components measured by the diagnostic survey 
were moderately correlated with students’ MAP ELA and math test scores in fall 2020 at the 
school level. In particular, home/online learning environment (ELA r = 0.42, math r = 0.45) and 
personal well-being (ELA r = 0.45, math r = 0.33) had the strongest associations with academic 
performance. Given the pandemic and challenges for young students to navigate online learning 
independently, it is not surprising that schools with young students who responded more 
positively about their home/online learning environment in spring 2020 tended to have higher 
average academic performance. Moreover, young students who reported that they felt safe at 
school (item 9; ELA r = 0.67, math r = 0.65) and that they did not procrastinate with schoolwork 
(item 15; ELAr = 0.42, math r = 0.43) also tended to have better academic performance. 


In middle schools, interpersonal well-being stood out as the most important component 
associated with students’ test scores (ELA r = 0.76, math r = 0.74). That is, middle school 
students who performed better academically also reported that they could more frequently find 
someone to talk to or that they had peer support when they were having a hard time. This was 
also supported by correlations with related individual items, as shown in Table 8. For example, 
middle school students who scored higher on the MAP test also responded more favorably 
to the items: “When you were learning from home this past spring, how often would you have 
someone available to help you with your schoolwork, if needed?” (ELA r = 0.53, math r = 0.45); 
“When | have a problem, | can come up with lots of ways to solve it” (ELA r = 0.72, math r = 0.58). 
Interestingly, middle school students’ personal well-being was negatively associated with test scores. 
One potential reason could be that students attending schools with relatively high academic 
scores may feel more pressure to perform well academically, and school closures may have 
exacerbated their concerns. It is unclear why middle school students’ home/online learning 
environment was negatively associated with the average test scores of these middle schools; one 
potential hypothesis is that students in high-achieving schools may have perceived their online 
learning environment as substantively worse than their typical school environment. 


Table 7. School-Level Correlations Between Students’ MAP ELA and Math Scores in Fall 2020 and 
Four Survey Topic Areas, by Grade Band 


[©] ¢-Te(-ef-lale| Number of Subject Personal Interpersonal Aol atoXo) a (Taaltare] a Koyaat-Vevaltlat-m(-t-laallate} 
schools well-being well-being Vai ceyaleat-tal' VaWViicelalaat-val' 

41 ELA 045 0.27 0.27 042 
Elementary 

41 Math 0.33 0.26 0.29 O45 

19 ELA -0.49 0.76 0.10 -0.36 
Middle 

20 Math -0.45 0.74 0.08 -0.32 
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Table 8. School-Level Correlations Between Students’ MAP ELA and Math Scores in Fall 2020 and 
Individual Items on the Survey, by Grade Band 


(eT ¢-\ol-M of-lale| Number = Subject Item 4 Item 6 Item 9 Item 15 Item 19 Item 20 Item 21 

of (hopeful) (solve (feel safe) (do work (someone (time spent on (like school) 

schools felgeye)(=104)) right away) EN ETI C1) (ce) schoolwork) 
rielt)) 

Al ELA 0.39 0.36 0.67 042 0.38 0.22 0.37 
Elementary 

Al Math 0.30 042 0.65 0.43 042 0.21 0.38 

19 ELA 0.22 0.72 0.45 -0.32 0.53 0.49 0.26 
Middle 

20 Math 0.30 0.58 0.38 -0.21 045 0.49 0.37 


Note. The full content of these individual items can be found in Appendix A. 


Next, we examined school-level correlations between students’ survey responses and their 
MAP ELA and math growth scores from fall 2019 to fall 2020. This analysis helped us understand 
whether students in high-growth and low-growth schools may have had distinct well-being 
and learning experiences. The correlations by grade band for the four topic areas are shown in 
Table 9; those for the seven individual items are shown in Table 10. These correlation coefficients 
required availability of MAP test scores in the last two fall administrations. Therefore, the number 
of schools was smaller than it was in Tables 7 and 8. The strength of association was also weaker. 
However, the overall pattern was similar. In middle schools (Table 9), students’ interpersonal well- 
being was not only an important component associated with students’ academic performance— 
it continued to be the most important component related to students’ academic growth during 
the school year affected by the pandemic and school closures (ELA r = 0.61, math r = 0.37). 


In elementary schools, associations between academic growth and survey responses 
were relatively weak but associations with growth in math were generally stronger than those 
with growth in ELA. In particular, not procrastinating with schoolwork (Table 10) had a much 
higher correlation with growth in math (item 15; ELA r = 0.09, math r = 044). 


Table 9. School-Level Correlations Between Students’ MAP ELA and Math Growth Scores 
(Fall 2019-Fall 2020) and Four Survey Topic Areas, by Grade Band 


(eT¢-lol-Mef-lale| Number of Subject Personal Tal e19 el-1ex0)ar-| | Tol aVoro) a (=r aal ale] a Koyant-VAevaltat-m(-t-laallate} 
Yel alolo) ky well-being well-being Vai celaleat-tal' VaWViicelaleat-tals 

36 ELA 0.14 0.01 0.13 0.13 
Elementary 

36 Math 0.21 0.11 0.20 0.22 

16 ELA -0.08 0.61 -0.14 -0.16 
Middle 

16 Math 0.11 0.37 -0.21 0.00 
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Table 10. School-Level Correlations Between Students’ MAP ELA and Math Growth Scores 
(Fall 2019-Fall 2020) and Individual Items on the Survey, by Grade Band 


(eT ¢-\el-M of-lale| Number = Subject Item 4 Item 6 Item 9 Item 15 Item 19 Item 20 Item 21 

of (hopeful) (solve (feel safe) (do work (someone (time spent on (like school) 

schools felgede)(=104)) right away) ENT ¢) (ce) schoolwork) 
rivet) 

36 ELA -0.16 -0.15 0.14 0.09 -0.09 -0.02 0.05 
Elementary 

36 Math 0.04 0.19 0.20 044 0.04 0.23 0.08 

16 ELA 0.13 0.28 0.09 -0.09 0.34 0.29 0.28 
Middle 

16 Math 0.16 0.25 0.21 0.11 0.12 -0.34 0.27 


Note. The full content of these individual items can be found in Appendix A. 


Fall 2020 versus Winter 2020-21 Survey Responses 


In addition to the diagnostic survey administered at the beginning of the school year, one 
CORE district also administered the comprehensive interim survey consisting of a set of mostly 
identical items in winter 2020-21. This section highlights findings from a total of 15,001 students 
who responded to both surveys. 


First, we compared students’ demographic characteristics and SBAC scores between 
those who only took the diagnostic survey in the fall (n = 23,121) and those who also took the 
comprehensive interim survey in the winter (n = 15,001). Table 11 shows that, among students 
who took both surveys, there were more girls, fewer English learners, and fewer students with 
disabilities. Moreover, a larger proportion of these students was proficient in the SBAC ELA and 
Math state summative tests in 2019. 
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Table 11. Percentage of Students in Each Subgroup Who Took the Fall Survey and Who Took 
Both Fall and Winter Surveys in Grades 4-12 


AYE] ote] Kole] e) Survey sample Difference 
Students who took the Students who took both 
fall survey fall & winter surveys 
Female 514 53.6 2.2 
Asian American 2.2 2.5 0.3 
Black 0.2 0.2 0 
Latinx 96.1 95.9 -0.2 
White 0.8 0.7 -O.1 
Economically disadvantaged 82.6 82.2 -O4 
English learner 26.7 23.5 -3.2 
Student with disabilities 8.0 6.8 -12 
SBAC 2019 ELA: standard met or exceeded 34.0 38.5 45 
SBAC 2019 math: standard met or exceeded 28.1 31.7 3.6 
Next, we compared the same students’ survey responses between fall 2020 and winter 


2020-21. Figure 13 shows that, after a few months of virtual learning in the 2020-21 school 
year, students’ home/online learning environment consistently improved across all grades. This 
improvement was larger among older students. Figure 14 further shows each of the three items 
included in this topic. Students were able to access online schoolwork more often (item 16), 
teachers provided online lessons more often (item 17), and students joined online learning lessons 
more often (item 18) in fall 2020 than in spring 2020. It is encouraging to see improvement in the 
accessibility and availability of online lessons as well as in students’ participation in these online 
lessons. The positive changes in students’ online learning environment over the course of the 
2020-21 school year is a testament to the herculean effort of educators, administrators, parents, 
and families to improve instructional experiences for students. As additional assessment data 
become available in the future, we hope to be able to examine whether students’ learning 
accelerates as remote and hybrid instructional practices continue to improve. 
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Figure 13. Mean Survey Responses Between Fall 2020 and Winter 2020-21 by Grade 
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Figure 14. Mean Responses of Items in the Home/Online Learning Environment Between 
Fall 2020 and Winter 2020-21, by Grade 


Item 16 (Access Online Schoolwork) Item 17 (Online Lessons Provided) Item 18 (Online Lessons Joined) 
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Unlike home/online learning environment ratings, students’ self-ratings in the other three 
topic areas of the surveys were similar between fall and winter overall (see Figure 13). However, 
these ratings were generally slightly lower in the winter among middle school and high school 
students, whereas ratings were slightly higher or similar in the winter among elementary students. 
Item 21 in Figure 15 offers a good representation of these patterns. On average, when asked, 

“In general, would you say that you like school?” at the beginning of the school year and then 
again a few months into the school year, students in Grades 5-12 responded with a much lower 
rating the second time (average difference is -0.62). The only exception is fourth graders, whose 
ratings increased by 0.26 on a O-4 scale. 


These findings about middle and high school students are concerning, and they echo 
other research that underscores the need to prioritize the mental health of children and 
adolescents given the effects of COVID-related school closures (e.g., Novins et al., 2021; 
Sommer, 2021). Our results further highlight the need for educators at all levels of the 
educational system to attend to the most pressing well-being and learning needs of each 
group of students at their unique developmental stage. 
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Figure 15. Mean Responses of Item 21 Between Fall 2020 and Winter 2020-21, by Grade 
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Takeaways 


This school year has been a challenging time for educators and students alike. Below, we 


highlight a few key takeaways from what we learned from analyzing students’ survey responses. 
We hope these lessons can help shed light on students’ well-being and help inform educators as 
they work to create a better learning environment for students from a whole child perspective. 


1. Overall, students rated their learning environment last year at school and last spring 
at home/online (2019-20) relatively highly and consistently across grades. However, 
students reported relatively lower ratings concerning their personal and interpersonal 
well-being. 


2. Students’ responses at the beginning of the 2020-21 school year varied across grades. 
Elementary students reported higher ratings on personal well-being and on school 
and home/online learning environment than did students in middle and high school. 
Older students reported higher ratings on interpersonal well-being. 
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3. Across four survey topics and grades, there were only a few Statistically significant 
differences in responses between economically disadvantaged students and their 
noneconomically disadvantaged peers. There were also very few statistically significant 
differences in responses among students in various racial/ethnic subgroups. However, 
we did discover distinct response patterns between boys and girls, EL and non-EL 
students, and students with and without disabilities. In addition, students attending 
schools that serve a higher percentage of students who receive FRL and students 
who are BIPOC reported that they spent less time each day on their schoolwork in 
spring 2020 and that they were less likely to access their online schoolwork. We urge 
educators, policymakers, and other stakeholders to pay special attention to the 
needs of English learners, students with disabilities, students who receive FRL, 
Black students, Indigenous students, and students of color, all of whom are more 
likely to be adversely impacted by the pandemic and distance learning. 


4. Connecting students’ well-being with their academic performance during the pandemic, 
we found moderate associations when aggregating data at the school level. In other 
words, schools at which students reported higher well-being and learning conditions 
tended to have higher average academic achievement. This school-level relationship 
also differed between elementary and middle school students. For older students, 
interpersonal well-being stood out as the most important area that was associated 
with both average academic achievement in fall 2020 and average academic growth 
between fall 2019 and fall 2020. 


5. Comparing students’ responses at the beginning of the 2020-21 school year and a few 
months later, in the winter, students’ home/online learning environment consistently 
improved across all grades. However, ratings from students in Grades 6-12 decreased 
in areas related to personal and interpersonal well-being as well as school learning 
environment. Given the continuation of mostly distance learning in California over this 
period, most students reported that they did not like school as much compared with 
how they felt at the beginning of the school year. 


6. Overall, results from the winter survey conveyed a similar message to those from the fall: 
students in higher grades reported lower personal well-being but higher interpersonal 
well-being compared to students in lower grades. However, the relatively strong 
association between interpersonal well-being and academic performance among 
older students highlights the needs of older students to maintain close relationships 
with their peers. Because these students are at a different developmental stage 
and are potentially more prone to social isolation, educators and parents should 
put additional effort towards creating an interactive learning environment for 
adolescent students. 
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Finally, we share the words students most commonly used when asked to describe 
themselves in three words (see Figure 16), as well as a few excerpts highlighting students’ voices 
in response to the question, “What makes you inspired to succeed?” In doing so, we hope to 
remind readers of the tenacity and strength students have shown, as well as the tremendous 
efforts of educators to overcome the challenges brought by the COVID-19 pandemic. 


Figure 16. Students’ Responses to “What Are Three Words That Describe You?" 
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“The things that make me inspired to succeed are my goals... Also, another thing 
that makes me inspired to succeed Is that if | keep my work ethic throughout my 
years of school, | am sure that | will be able to go to any college of my choice." 


‘| find that having a teacher who shows that they're facing the same struggles and 
understands those of their students really helps to encourage/inspire me to work 
Just as hard if not harder.” 


“Listening to talks about our future like the one Ms. R. gave us today. | honestly 


haven t been feeling motivated at all about life in general and mostly about my 
school but Ms. R. really motivated me to get myself together and chase my dream." 
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Appendix A. Student Well-Being and Learning Conditions 


Diagnostic Survey 


This survey asks about how you are doing, as well as your behavior, experiences, and 
attitudes related to school. We look forward to using your feedback to make the start of your 
school year better. Some of the survey questions will ask you about specific periods of time 
(such as in the past week, last year at school, or after school closed last spring). Please pay careful 
attention to these time periods when you respond. Thank you for taking this survey! 


Part 1: Now or the Past Few Days 


For the questions in this section, please resoond based on your current experiences or in the 
recent past (last few days): 


DOAWNE 


During the past week, how often did you feel happy? 
During the past week, how often did you feel worried? 
During the past week, how often did you feel sad? 
During the past week, how often did you feel hopeful? 
When | need help, | find someone to talk to. 


When | have a problem, | can come up with lots of ways to solve it. 


(Never, Once in a while, Sometimes, Frequently, Always) 


Are there students at your school who would help you if other students are being mean to you? 
Do you have a friend your age who helps you when you are having a hard time? 


(Almost never, Once in a while, Sometimes, Often, Almost all the time) 


Part 2: Last Year at School 


For the questions in this section, please respond based on your experiences last year in school 
(when you were at school): 
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ur teachers treat you with respect? 
ur teachers care what you thought? 


id teachers go out of their way to help students? 


ur teachers work hard to help you with your schoolwork when you 


u get your work done right away instead of waiting until the last 


(Almost never, Once in a while. Sometimes, Often, Almost all the time) 
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Part 3: Learning from Home This Past Spring 


For the questions in this section, please resoond based on when you were learning from home 
this past spring: 


16. When you were learning from home this past spring, how often were you able to access your 
schoolwork that was online? 

17. When you were learning from home this past spring, how often did your teacher have online 
lessons with the class? 

18. When you were learning from home this past spring, how often did you join online learning 
lessons with your teacher? 

19. When you were learning from home this past spring, how often would you have someone 
available to help you with your schoolwork, if needed? 


(Almost never, 1 day per week, 2-3 days per week, 4 days per week, 5 days per week) 


20.When you were learning from home this past spring, about how much time did you spend 
each day on schoolwork? 


(O hours, 1 hour, 2-3 hours, 4-5 hours, More than 5 hours) 


Part 4: Tell Us About Yourself 
For the questions in this section, please resoond based on how you feel now: 
21. In general, would you say that you like school? 
(Not at all A little, Somewhat, Pretty much, Very much) 
22. Which type of adults have helped you the most in the past? (Check all that apply.) 


(Teacher, Counselor, Therapist, Parent, Guardian, Sibling, Other family member, Coach, 
Mentor, Spiritual leader, Social worker, No one, Other) 


25. How do you typically form relationships with adults? 


(I rarely have relationships with adults; | wait for adults to begin a relationship with me; 
Sometimes | begin relationships with adults and sometimes wait for others to do it; 
! begin relationships with adults easily) 


24. What are three words that describe you? 
25. Can you tell us about a time when an adult at school really helped you? 
26. How do you like to spend your free time? 


(Free response) 
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Appendix B. Supplemental Analysis Results 


Table B1. Student-Level Correlations Between Students’ MAP ELA and Math Scores in Fall 2020 
and Four Survey Topic Areas, by Grade Band 


(eT ¢-lel-M of-lale| Maximum Subject Personal Tale el-1ex0)ar-| | Kol tote) (=F Taaliale] a Koyant-VAovaltlat-m(-t-laallate} 
number of W(=11G eY-1 are] well-being Vail ceyaleat-tal' VaWViicelaleat-tals 
students* 

6,767 ELA 0.12 0.09 0.15 0.21 

Elementary 

6,765 ath 0.15 0.05 0.11 0.22 
6,590 ELA -0.06 0.11 0.04 -0.07 
Middle 
6,511 ath -0.01 0.13 0.05 -0.08 
2,596 ELA -0.12 0.13 0.07 -0.04 
High 
2,392 ath -0.09 0.13 0.07 -0.03 
* The numbers of students are slightly different between different pairs of MAP subject — survey topic correlations. The maximum 
number of students is reported in Tables B1-B4. 


Table B2. Student-Level Correlations Between Students’ MAP ELA and Math Scores in Fall 2020 
and Individual Items on the Survey, by Grade Band 


(eT ¢-lol-M ef-lale| Maximum Subject Item 4 Item 6 Item 9 Item 15 Item 19 Item 20 Item 21 
ralelaal eX-lae) i (hopeful) (solve (feel safe) (do work (someone (time spent on (like 
students* fede) olt-1a1)) right away) _—_ available to schoolwork) school) 

help) 
6,767 ELA 0.03 0.10 0.15 0.21 0.09 -0.02 0.11 

Elementary 

6,765 ath 0.02 0.11 0.13 0.20 0.06 -0.02 0.11 

6,590 ELA -0.01 0.08 0.07 0.05 0.05 -0.01 0.06 
Middle 

6,511 ath 0.02 0.11 0.09 0.07 0.03 0.002 0.08 

2,596 ELA -0.07 0.08 0.08 -0.03 0.09 0.04 0.04 
High 

2,592 ath -0.03 0.09 0.07 0.00 0.09 0.06 0.06 


Note. The full content of these individual items can be found in Appendix A. 
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Table B3. Student-Level Correlations Between Students’ MAP ELA and Math Growth Scores 
(Fall 2019-Fall 2020) and Four Survey Topic Areas, by Grade Band 


(eT ¢-\ol-M oF-lale| Maximum Subject Personal Tal 19 eX-1exe)ar-|| Yel aCoro) a (=r aaliale] a Koyant-Vevaltlat-m(-t-laallate} 
alvlanlel-1meyi well-being well-being Vai ceyaleat-vals VaWViicelaleat-tal' 
students* 

6,526 ELA 0.08 0.04 0.07 0.08 

Elementary 

6,518 ath 0.08 0.03 0.04 0.06 
6,235 ELA 0.01 0.05 0.05 -0.03 
Middle 
6,166 ath 0.02 0.08 0.06 -0.01 
2,320 ELA -0.02 0.05 0.08 0.00 
High 
2,103 ath -0.04 0.04 0.02 0.00 
Table B4. Student-Level Correlations Between Students’ MAP ELA and Math Growth Scores 
(Fall 2019-Fall 2020) and Individual Items on the Survey, by Grade Band 


(eT ¢-lel-Mef-lale| Maximum Subject Item 4 Item 6 Item 9 Item 15 Item 19 Item 20 Item 21 
ralelaaleysl ane) i (hopeful) (solve (feel safe) (do work (someone (time spent on (like 
Frattet-\ al total problem) right away) _— available to schoolwork) school) 
help) 
6,526 ELA 0.02 0.04 0.08 0.07 0.07 0.01 0.02 
Elementary 
6,518 ath -0.004 0.02 0.03 0.07 0.05 0.001 0.003 
6,235 ELA 0.003 0.03 0.04 0.04 0.03 0.01 0.06 
Middle 
6,166 ath 0.02 0.05 0.06 0.04 0.003 0.02 0.07 
2,520 ELA -0.01 0.03 0.08 0.06 0.05 0.10 0.04 
High 
2,103 ath -0.03 0.02 -0.01 0.03 0.05 0.08 0.01 


Note. The full content of these individual items can be found in Appendix A. 
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Table B5. Fall 2019-to-Fall 2020 Growth in MAP ELA Scores (and Number of Students) 
for Students in the Bottom Quartile (Q1) and Top Quartile (Q4) of Each Survey Topic, 
by Demographic Subgroups 


AYE] ole] colt] e) ‘Le 2. 3. 4. 
Personal well-being Interpersonal NYol afoXo) MKF Taal lars] a Coyant-VAeyaltat-m(-t-laallate} 
well-being Vai icedaloat-als Vai icelalaalsals 
cele Taal() Qi Q4 Qi Q4 Qi Q4 Q1 Q4 
Female -0.72 0.12 -1.14 0.30 -0.96 -0.04 -0.25 -0.12 
(2,142 1,650 (1,156) (2,394) 1,664) (2,345) (1,541) (2,637) 
BIPOC -121 -0.62 -1.66 -0.19 -1.78 -0.34 -0.68 -0.43 
(3,448 3,968 (2,758) (4,052) 3,631) (4,090) (2,912) (5,179) 
Economically -1.24 -0.70 -1.78 -0.12 -1.85 -0.33 -0.85 -045 
disadvantaged (2,963 3,379 (2,359) (3,411) 3,133) (3,537) (2,494) (4,394) 
English -2.67 -144 -248 -0.95 -2.71 -0.93 -247 -0.87 
learners (1,000 1,226 (969) (958) 1,126) (1,170) (836) (1,409) 
Students with -2.67 -1.67 -3.02 -151 -3.64 -1.02 -2.69 “1.25 
disabilities (275) (313) (315) (218) (347) (298) (251) (366) 
All students -1.20 -0.62 -1.64 -0.19 -1.74 -0.33 -0.66 -0.42 
(3,482) (3,994) (2,780) (4,080) (3,669) (4,118) (2,936) (5,218) 
Table B6. Fall 2019-to-Fall 2020 Growth in MAP Math Scores (and Number of Students) 


for Students in the Bottom Quartile (Q1) and Top Quartile (Q4) of Each Survey Topic, 
by Demographic Subgroups 


AYE] ote] cole] 8) 1. 2. 3. 4. 
=) e-Xo)al-] MW(=1( G9 ol-1 ale] Interpersonal Nol aToXo) MI Ta alias] a Koyaat-VAevaltat-m(-t-laallare} 
well-being Va icedalaa-als Vai icelalaal-als 
Quartile Qi Q4 Q4 Qi Q4 Qi Q4 
Fecale -0.22 -0.36 -1.12 040 -040 -0.17 0.33 -0.09 
(2,104) (1,627 (1,144) (2,372) (1,641) (2,330) (1,476) (2,626) 
BIPOC -0.76 -0.57 -149 0.04 -1.14 -0.35 -0.34 -0.24 
(3,372) (3,891 (2,702) (3,989) (3,554) (4,048) (2,761) (5,143) 
Economically -0.88 -0.49 -144 0.02 -1.16 -0.32 -0.37 -0.25 
disadvantaged (2,891) (3,309 (2,318) (3,358) (3,062) (3,505) (2,369) (4,356) 
English -1.80 -0.71 -1.64 -0.31 -154 -0.70 aLiy -0.45 
learners (989) (1,200 (947 (966) (1,120) (1,157) (832) (1,397) 
Students with -2.55 -1.37 -1.65 =1.15 -2.28 -0.87 -1.24 -0.05 
disabilities (264) (310) (315 (218) (342) (299) (237) (369) 
All students -0.79 -0.58 -1.52 0.03 -1.18 -0.35 -0.35 -0.26 
(3,402) (3,919) (2,725) (4,017) (3,592) (4,078) (2,781) (5,184) 
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Figure B1. Mean Scores of Main Survey Topics for Asian American and White Students, by Grade 
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Note. ns = not significant (p-value > 0.05); * p-value < 0.05; ** p-value < 0.01. The vertical black bars represent the 95 percent 
confidence interval of the difference in mean scores. 
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Figure B2. Mean Scores of Main Survey Topics for Black and White Students, by Grade 


w 
A 


Nn 
n 


ray 
4 


sco. 


Mean 


N Ww 
4 n 


ra 
L 


2 of inn 


1. Personal Well-Being 2. Interpersonal Well-Being 


—a 


3. School Learning Environment 4. Home/Online Learning Environment 


4+ ns ns ns ns ns ns ns ns 


a a A A SR A GA" nn 


ns 


” ‘Genlle 


B Black @ White 


Note. ns = not significant (p-value > 0.05). The vertical black bars represent the 95 percent confidence interval of the difference in 


mean Scores. 
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Figure B3. Mean Scores of Main Survey Topics for Latinx and White Students, by Grade 
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Note. ns = not significant (p-value > 0.05); * p-value < 0.05; ** p-value < 0.01. The vertical black bars represent the 95 percent 
confidence interval of the difference in mean scores. 
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Figure B4. Fall 2019-to-Fall 2020 Growth in MAP ELA and Math Scores Between Students in the 
Bottom Quartile (Q1) and Top Quartile (Q4) of Personal Well-Being, by Grade 


Mean MAP growth estimates 


Figure BSB. Fall 2019-to-Fall 2020 Growth in MAP ELA and Math Scores Between Students in the 
Bottom Quartile (Q1) and Top Quartile (Q4) of Interpersonal Well-Being, by Grade 
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Figure B6. Fall 2019-to-Fall 2020 Growth in MAP ELA and Math Scores Between Students in the 


Bottom Quartile (Q1) and Top Quartile (Q4) of School Learning Environment, by Grade 
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Figure B7. Fall 2019-to-Fall 2020 Growth in MAP ELA and Math Scores Between Students in the 
Bottom Quartile (Q1) and Top Quartile (Q4) of Home/Online Learning Environment, by Grade 


bo ns ns ns ns ns ns 
24 
0 m 
5 

nn 
vo 
2 
oO 
£2) 
S 
a 
o 
8 | 
2 
00 ns ns ns ns ns ns 
a 
<= 
= 27 
= 
o 
vo 
= 


° 
HLVW 


Student Well-Being and Learning Conditions During the Pandemic: Evidence from the CORE Districts 


Policy Analysis for California Education 


Author Biographies 


Caroline Wang is a research scientist at Education Analytics, a nonprofit organization based in Madison, 
Wisconsin, that conducts statistical research to solve complex, pressing problems facing public education in the 
United States. She received her PhD in educational research, measurement, and evaluation from Boston College. 


Libby Pier is the director of impact at Education Analytics. She previously taught middle and high school English 
in Los Angeles through Teach For America. She received her PhD in learning sciences from the Department of 
Educational Psychology at the University of Wisconsin—Madison. 


Robert H. Meyer is the president and CEO of Education Analytics. He was formerly the director of the Value- 
Added Research Center at the University of Wisconsin—Madison and has conducted major statistical evaluations of 
programs and policies, such as class-size reduction, literacy, and supplemental educational services. He received 
the University of Wisconsin—Madison Chancellor's Award for Excellence in Research in 2010. 


Nichole Webster is a research analyst at Education Analytics. She examines the item properties and performance 
of social-emotional learning surveys and estimates teacher and school performance metrics in the software 
environment R. She received her MS in applied economics from the University of Wisconsin—Whitewater. 


edpolicyinca.org © 


SAOLUCOAVAAVAT-1NVA1I Ms ke) am @r- 1 0hke) a alt- make |Uer- hale) am( A\@l =) 


Improving education policy and practice and advancing equity 
through evidence 


PACE Is an independent, non-partisan research center led by faculty directors at 
Stanford University, the University of Southern California, the University of California 
Davis, the University of California Los Angeles, and the University of California 
Berkeley. Founded in 1983, PACE bridges the gap between research, policy, and 
practice, working with scholars from California’s leading universities and with 

state and local decision makers to achieve improvement in performance and 
more equitable outcomes at all levels of California's education system, from early 
childhood to postsecondary education and training. We do this through: 


ch bringing evidence to bear on the most critical issues facing our state; 
€ making research evidence accessible; and 
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